PROCEDURAL LEARNING 1 Running head: Procedural learning Structuring Information Interfaces For Procedural Learning

Interface design should be informed by the application of top-down cognitive principles derived from basic theory and research. We applied cognitive design principles from two domains, event cognition and media, to the design of interfaces for teaching procedures. According to theories of event cognition, procedures should be presented hierarchically, organized by objects or large object parts and actions on objects. According to research on effects of media, adding appropriate graphics to text instructions can facilitate learning and memory. These principles were partially supported in two tasks: assembling a musical instrument and building a model. Although both top-down principles were effective in guiding interface design, they were not sufficient. They can be combined with iterative bottom-up methods to produce usable interfaces. PROCEDURAL LEARNING 3 Introduction Designing Interfaces. There are two ways to design human-computer interfaces: top-down and bottom-up. The top-down method applies general principles derived from cognitive research to a specific interface for a particular task (e.g., Norman, 1988; Shneiderman, 1998). The bottom-up method analyzes the structure of the particular task, varying its’ features systematically to determine their optimal values and refining by iterated testing and design (e.g., Egan, Remde, Landauer, Lochbaum, & Gomez, 1995; Nielsen, 1993). The first method has obvious advantages: It takes a small number of general principles derived from basic research on human cognition and applies them to myriad and numerous domains. Thus it both encourages basic research and raises the promise of wide applicability. Despite these advantages, the top-down method has inevitable limitations and gaps for the design of interfaces in specific cases (Carroll, 1991; Landauer, 1991). General principles are too general to guide the specific design decisions that may ultimately determine the success of an interface. Furthermore, general principles are typically not quantifiable; as such, they do not inform the trade-offs that are an essential part of interface design. These considerations suggest that an ordered hybrid approach is the best: Use the top-down principles to bound the space of possible interfaces and to suggest promising directions; then use the bottom-up approach to refine specific cases. One common application whose interfaces all too often draw the ire and sighs of users is instructions to assemble an object or operate a device. Such tasks require conveying a continuous sequence of complex actions, and raise questions about the sequence and about how to convey it. Two domains of basic cognitive research may inform interface design for that needy domain: research on event cognition and PROCEDURAL LEARNING 4 research on media. Research on event cognition provides information about how people conceive of sequences of actions and research on media provides information about the modalities effective in conveying different kinds of information. We applied the top-down approach to the design of interfaces for procedural learning. The case we chose to study was object assembly, a task familiar to our population of college students and legendary for inadequate instructions. Designing an effective interface for object assembly has two major components: schematizing the continuous action of assembly and using the media of language and depiction to convey the procedures. The cognitive principles we applied were derived from our own and others’ research in two domains: event cognition and effects of various media. We chose two different and representative test applications for these principles, assembly of a musical instrument and assembly of a toy. The interfaces implementing the principles were designed with care, and the detailed evaluation of performance revealed both the benefits and the limitations of the top-down method. The limitations, felicitously, revealed general principles of their own. We measured both immediate performance of the task to be learned and subsequent memory for the steps involved, because although on some occasions it is important simply to be able to complete an assembly task, at other times the task must be learned for later performance. Event Cognition. Assembly is a common activity in our daily lives, from putting on clothes and fixing a meal to putting together the new equipment in the office or the piece of furniture. Children, too, spend hours assembling things, from nesting cups to Lego palaces. Assembly is a paradigm case of a complex event, an organized sequence of behaviors that has a beginning, middle, and end. Central to events are achievements or accomplishments, such as climbing a mountain or knitting a sweater (see Zacks & Tversky, 2001 for a review). Despite the fact that events are continuous series of actions, PROCEDURAL LEARNING 5 people conceive of them as discrete sequences. When asked to segment events into the largest and smallest units that make sense, people do so hierarchically; that is, the large event boundaries coincide more often than chance with the small event boundaries, indicating that people regard the fine units as components of the coarse ones (Newtson, 1973; Zacks, Tversky, & Iyer, 2001). What underlies the boundaries between event segments? People’s descriptions of what occurred in each segment provides insight into that. More than 90% of the descriptions were actions on objects, that is, goaldirected behaviors, such as “put on the top sheet” or “rinse the plate” (Zacks et al., 2001). What’s more, there were qualitative differences between descriptions of coarse and fine segments. Specifically, large event segments were punctuated by separate objects or large object parts whereas fine event segments were distinguished by different actions performed on the same part or object. According to the Principle of Congruence, external representations such as instructions for assembly should conform to desired internal representations, other things being equal (Tversky, Morrison, & Betrancourt, in press). Applying cognitive event structure to the design of an interface for object assembly insures that the instructions will be compatible with users’ mental representations of assembly, enhancing their comprehension. Considerations of cognitive event structure imply that instructions should be segmented and hierarchical and should break the sequence where people do, around objects or large object parts at the coarse level and around articulated actions on objects at the fine level. Hierarchical presentation of instructions has proved to be beneficial in a number of contexts, for example, in comprehending instructions for assembling circuits (Smith & Goodman, 1984), operating machines (Dixon, 1987b), and drawing simple figures (Dixon, 1987a; Dixon, Faries, & Gabrys, 1988). PROCEDURAL LEARNING 6 Media. The media of instructions as well as their structure affect their transparency and have consequent effects on performance. According to the Principle of Apprehension (Tversky et al., in press), external representations should be readily perceived and accurately conceived. From general research in cognition, it is known that memory for pictures is superior to verbal presentation of the same material (for a summary, see Paivio, 1986) One account of this widespread phenomenon is that pictorial codes provide more and richer retrieval cues. An additional advantage of pictorial presentation for concrete procedures is that it can directly portray the procedures rather than explaining them in language; it uses depictions of action to convey action. Mapping the spatial to space is compelling and natural in the sense that children and adults in communities all over the world have created such mappings (e.g., Tversky, 2001). In addition, presenting material in two media, pictorial and verbal, is generally superior to presenting material in only a single medium, though of course this depends on the pictorial information being well designed and integrated (Mayer, 2001). In particular, when combining graphics and text it is important that the two be integrated to avoid problems due to splitting attention (Sweller, 1999). Considerable research supports the efficacy of graphic depictions in various applied settings (for reviews, see Levie & Lentz, 1982; Levin, Anglin, & Carney, 1987; Mayer & Gallini, 1990; Winn, 1989). In a recent review, Mayer and Moreno (2002) summarized seven prescriptions for the design of educational interfaces using animation. At the top of their list was the multimedia principle, which says that deeper learning results when animation is combined with text. Animated pictures seem a natural extension of enrichment of stimuli, from words to pictures to moving pictures. Animations have a further possible benefit for assembly in cognitive compatibility: They use change over time to convey change over PROCEDURAL LEARNING 7 time. However, animations have disadvantages as well. Animations can be complex, and viewers may not know where to focus attention. This complexity renders them difficult to process and consequently, difficult to remember. Animations are fleeting and cannot be inspected and reinspected as static graphics can, without special interface support and explicit user action. Perhaps for these reasons, and despite their natural correspondence, in practice, animations have not yielded better performance than informationally equivalent static diagrams in a large variety of contexts (see Tversky et al., in press for a review). Nevertheless, because of the natural correspondence of using portrayals of change over time to convey concepts of change over time, hope remains that the proper animation for the proper domain should show benefits. For conveying events, animated graphics have potential advantages and disadvantages. Although they use change over time to convey change over time, they are continuous, whereas people conceive of events as discrete, so animations do not correspond to the way people conceive of events. On the other hand, animations can convey manner and timing of action, which when complex or subtle, as in knot tying or tennis serves, can be difficult to portray in static graphics. The top-down approach to designing interfaces for assembly, then, provides two cognitive design principles: First structure the interface to match the structure of the procedure to be taught. This means the interface should explicitly represent the hierarchical structure of the task to be performed. Second, use text and pictures, with a question mark on animated vs. static pictures. We applied these principles to the design of interfaces for two different assembly tasks, assembly of a saxophone and assembly of a toy bug from a construction kit. For the saxophone, we had determined the hierarchical structure from previous research (Zacks et al., 2001), but we doublechecked and refined the hierarchy for this experiment. For the toy assembly, we built in PROCEDURAL LEARNING 8 the hierarchical structure into instructions. The tasks differ in several ways. Although order of assembly is constrained in both, that is, certain operations must be accomplished before others, there are fewer constraints in the toy assembly. Experiment 1: Assembling a Saxophone In the first experiment we taught participants how to put together a tenor saxophone, as if to play it. This activity was chosen because we had normative data regarding how people perceive it and because it is unfamiliar to most of our participants. Although there are minor variations in how one orders the steps involved in this task, we chose one particular ordering as the target for training (see Table 1). This small constraint on the naturally occurring activity was intended to capture the common situation in which the affordances of the parts to be assembled permit multiple orders, but only a subset of these are correct. We refer to this as the “Ikea effect,” (after the furniture company): the experience of coming to the end of a complex assembly task only to discover that there are still critical parts left over, or that the partially assembled pieces don’t fit together because a step was performed too soon. Restricting the order of performance also allowed for precise quantification of how well participants learned order information. Participants learned from a computer interface that varied on two dimensions. First, we varied the media in which the instructions were presented: text, text plus still pictures, or text plus video. Second, we varied the layout of the interface, presenting the steps organized hierarchically based on our normative perceptual data or simply as a list. We hypothesized that adding visual depictions to the text descriptions would improve memory for the task instructions, and might thereby improve the quality of their performance of the task. We also hypothesized that structuring the interface in PROCEDURAL LEARNING 9 accord with normative conceptual representations would improve participants’ ability to learn and remember the temporal order of the task. We predicted that that this would improve their ability to perform the parts of the task in the correct order and later remember that order.